Skip to content

Comments

added why.md for the environments#23

Open
yogesh1801 wants to merge 98 commits intosys-intelligence:mainfrom
yogesh1801:pr/why-md
Open

added why.md for the environments#23
yogesh1801 wants to merge 98 commits intosys-intelligence:mainfrom
yogesh1801:pr/why-md

Conversation

@yogesh1801
Copy link
Collaborator

@yogesh1801 yogesh1801 commented Nov 27, 2025

Description

This PR addresses Issue #22 by adding dedicated WHY.md files to each benchmark directory and linking them from the root README. These files explain why each benchmark matters and how it fits into the broader vision of system intelligence, following the pattern established in PR #21.

Changes

  • Added WHY.md to System Exam Benchmark
  • Added WHY.md to System Lab Benchmark
  • Added WHY.md to System Artifact Benchmark
  • Added WHY.md to System Modeling Benchmark
  • Added WHY.md to Cache Algorithm Benchmark
  • Updated root README.md to add WHY links next to each benchmark entry in the benchmark list
  • Added Cache Algorithm Benchmark to the root README benchmark list (was previously missing)

Testing

  • Verified all 5 WHY.md files exist in their respective benchmark directories
  • Confirmed all WHY.md links in root README.md point to correct file paths
  • Reviewed each WHY.md for consistency with benchmark READMEs and overall system intelligence vision
  • Validated markdown formatting renders correctly

Checklist

  • Tests pass locally (documentation-only changes)
  • Code follows project style guidelines
  • Documentation updated (this PR is documentation enhancement)

xuafeng and others added 30 commits November 5, 2025 18:10
@xuafeng
Rename it "System Intelligence Benchmark"
6d24e69
@Qian-Cheng-nju
Init: Initialize SysMoBench benchmark integration
87db1e9
@Qian-Cheng-nju
feat: Add gitigore
69f4cb5
@Qian-Cheng-nju
feat: Add prototype for phase 1&2
843f031
@Qian-Cheng-nju
feat: Distinguish evaluator and model API keys in env.toml
0d2b38f
@Qian-Cheng-nju
feat: Add validation for required evaluator API keys
b2acaa7
@Qian-Cheng-nju
doc: update README.md
ca7e72e
@bastoica
initial ArtEval commit
ec7b57f
@xuafeng
Merge pull request sys-intelligence#2 from systemintelligence/feat/di…
a607e73 
…stinguish-api-keys

Distinguish the models used in the executor and evaluator
@Qian-Cheng-nju
feat: Add test
60d30e0
@Qian-Cheng-nju
featr: Add install.sh
ff96313
@bastoica
adding overview and contributor's guide
1799370
@bastoica
skeleton ArtEval agent implementation
2054314
@bastoica
adding sosp24 wasabi
6303aa5
@tareknaser
docs: add arteval to main README
a5358dc 
Signed-off-by: Tarek <tareknaser360@gmail.com>
@tareknaser
feat(ci): add GH Actions workflow for running benchmarks tests
904374e 
Signed-off-by: Tarek <tareknaser360@gmail.com>
@tareknaser
feat: add issue and pull request templates
40ccf1f 
Signed-off-by: Tarek <tareknaser360@gmail.com>
@tareknaser
fix(ci): add a test for example_bench
4130c7a 
Signed-off-by: Tarek <tareknaser360@gmail.com>
@tareknaser
fix: shell scripts to be executable
3af5b70 
Signed-off-by: Tarek <tareknaser360@gmail.com>
@tareknaser
docs: update README with instructions for running a single benchmark
156c77c 
Signed-off-by: Tarek <tareknaser360@gmail.com>
@tareknaser
docs: a note on docker image arch support
a0557f9 
Signed-off-by: Tarek <tareknaser360@gmail.com>
@tareknaser
meta: add outputs directories to gitignore
868da59 
Signed-off-by: Tarek <tareknaser360@gmail.com>
@tareknaser
feat(ci): add release trigger to workflow
ea9b54d 
Signed-off-by: Tarek <tareknaser360@gmail.com>
@Qian-Cheng-nju
fix: Use tla_specification instead of generated_text to adapt upstrea…
5ef835f 
…m changes
@Qian-Cheng-nju
Merge commit '04900168e10834f3aa5eef4d13b318e1efcdac24' as 'benchmark…
c5dfbb1 
…s/sysmobench/sysmobench_core'
@Qian-Cheng-nju
fix: Add gpt-4o config and fix cross-device link issue in setup_tools
a68e171 
- Add gpt-4o model configuration to models.yaml
- Fix setup_tools.py to use shutil.move instead of os.rename
  This resolves 'Invalid cross-device link' error when /tmp is on different filesystem
@Qian-Cheng-nju
fix: Convert GenerationOutput to GenerationResult for evaluators
984336a
@Qian-Cheng-nju
docs: Update README and install script for Git Subtree integration
68025ca
@Qian-Cheng-nju
feat: Add docker file
25c6af8
@Qian-Cheng-nju
fix: Add env.toml
97c1c3c
xuafeng and others added 17 commits November 19, 2025 08:59
@xuafeng
Merge pull request sys-intelligence#16 from sys-intelligence/docs_cou…
f7d3cbd 
…rse_lab_bench

Course Lab Benchmark: Add Instructions for Extending the Benchmark
@xuafeng
Merge pull request sys-intelligence#15 from sys-intelligence/arteval_…
535a78f 
…benchmark

Adding ArtEvalBench v0.9
@bastoica
refactor: reorganizing and improving the step-by-step guidelines for …
e6361a4 
…adding new artifacts to ArtEvalBench
@bastoica
fix: add a brief explanation for 'docker_env' schema field
2523106
@bastoica
refactor: update the new schema naming convention
e0059f3
@bastoica @tareknaser
fix: apply suggestions to WHY.md
c72230a 
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
@bastoica @tareknaser
fix: a few typos in WHY.md
77b6e7f 
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
@bastoica
refactor: minor formatting and style improvements
c207e59
@bastoica
fix: remove obsolete dependency installation instructions from README
db6a947
@bastoica
refactor: rework the first paragraph and fix minor text redering issues
8b05a67
@bastoica
fix: clean-up repository, remove unnecessary or unused scripts
07f76c0
@bastoica
fix: update Dockerfile and removed unused scripts
16c35b0
@bastoica
feature: add updated Docker image and environment bootstrapt scripts
f73cb6a
@bastoica
fix: few tweaks re installation and setup
44c37be
@bastoica
refactor: add default Docker image, rewrite agent prompt, and remove …
2a8d374 
…ureachable code
@bastoica
fix: patch the benchmark schema file and improve Wasabi's README inst…
6409d2f 
…ructions
@xuafeng
Merge pull request sys-intelligence#21 from bastoica/main
13c5fa1 
Improving the "contributor's guide" and simplifying the benchmark's schema
@bastoica bastoica self-requested a review December 6, 2025 21:25
bastoica
bastoica requested changes Dec 6, 2025
View reviewed changes
Copy link
Collaborator

@bastoica bastoica left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused as why you added a new WHY.md for arteval_bench. Is this simply the existing file or did you make any edits? Thanks!

@yogesh1801
Copy link
Collaborator Author

yogesh1801 commented Dec 6, 2025 •
edited
Loading

Hi @bastoica sorry for the confusion the arteval benchmark file is same, it is error from my side that it looks like a new file in commit, but it is the same
I have fixed the issue in the next commit

@yogesh1801
why md for benchmarks
3928b04 
Signed-off-by: Yogesh <yogeshsingla481@gmail.com>
@bastoica bastoica self-assigned this Dec 6, 2025
@bastoica bastoica requested a review from xuafeng December 6, 2025 23:00
@bastoica
Copy link
Collaborator

bastoica commented Dec 6, 2025

sounds good, thanks @yogesh1801

@xuafeng
Copy link
Collaborator

xuafeng commented Dec 11, 2025

@tareknaser can you help review if the new WHY.md make sense to you?

@xuafeng
Copy link
Collaborator

xuafeng commented Dec 11, 2025

@Qian-Cheng-nju can you help review if the new WHY.md of SysMoBench works for you? Welcome any comments.

@Qian-Cheng-nju
doc: Revise benchmark system count and types in WHY.md
c783e5c 
Updated the number of systems and their types in the benchmark description.
@Qian-Cheng-nju
Copy link
Collaborator

Qian-Cheng-nju commented Dec 12, 2025

@Qian-Cheng-nju can you help review if the new WHY.md of SysMoBench works for you? Welcome any comments.

We recently added the ringbuffer module from Asterinas and ZooKeeper, so I updated the description of the number and types of systems. Everything looks good to me now. Thank you very much for such a detailed document!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bastoica bastoica bastoica requested changes

@xuafeng xuafeng Awaiting requested review from xuafeng

Assignees

@bastoica bastoica

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@yogesh1801 @bastoica @xuafeng @Qian-Cheng-nju @tareknaser